DOCUMENT RESUME 



ED 233 045 



TM 830 473 



AUTHOR 
TITLE 

INSTITUTION 

PUB DATE 
NOTE 



PUB TYPE 



EDRS PRICE 
DESCRIPTORS 



IDENTIFIERS 



Holley, Freda M. 

Is There Evaluation after Federal Cutbacks? 
Publication No. 82.62. 

Austin Independent School District, Tex. Office of 
Research and Evaluation, 
Apr 83 

12p.; Paper presented at the Annual Meeting of the 
American Educational Research Association (67th, 
Montreal, Quebec, April 11-15, 1983). 
Speeches/Conference Papers (150) — Reports - 
Research/Technical ( 143 ) 

MFOl/PCOl Plus Postage. 

*Compensatory Education; *Hducational Finance; 
*Evaluation ; *Federal Programs; *National Surveys ; 
School Districts; * School Surveys ; Urban Areas 
Cutback Management; *Elementary Secondary Education 
Act Title I 



ABSTRACT 

To ascertain how evaluation has fared in the 
eighties, the Austin Independent School District surveyed the 94 
largest school districts (including Hawaii and Puerto Rico) shown in 
the Directors of Research and Evaluation Network directory. A usable 
return rate of 63 percent was received for data analysis. As 
expected, the overall number of students eligible for Title I 
services increased as the overall enrollments in large cities 
continued to decrease. The average program budget decreased along 
with the average Title I evaluation budget. The average number of 
clerical, professional, and administrative staff declined. A smaller 
percentage decrease in overall federal funds for evaluation looks as 
though Title I funding may be taken up by other sources of federal 
monies. A surprise comes in the increase in local dollars for 
evaluation. It would appear that local school systems do value the 
evaluation services they are receiving. The survey, form and district 
responses to the question of whether there were changes in evaluation 
due to the reduction of federal funds are appended. (PN) 
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IS THERE EVALUATION AFTER FEDERAL CUTBACKS? 



Freda M. Holley 
Austin Independent School District 

Along with compensatory education, the Elementary and Secondary Ed- 
ucation Act of 1965 introduced research and evaluation units into numerous 
school systems around the country. In a very real sense, evaluation's 
testing period was the 1970's and the report on that trial is now coming in. 
Do research and evaluation contribute to school systems at levels that jus- 
tify their cost in local dollars? When federal dollars become scarce and 
the level of evaluation optional will evaluation be cut? 

In my own mind there are many parallel ways to think about compen- 
satory education and research and evaluation. Both sets of activities are 
almost moral imperatives in contemporary American education. Too, my own 
experience in education is inextricably tied to both. My first job was 
teaching minority children in a Texas school district. I learned more that 
year than my students. I'm afraid. For example, as an inexperienced, un- 
certified teacher in 1973 . I was placed in the most difficult of teaching 
situations - the lower level of two tracked classes at 5th grade in that 
ghetto school, with 42 children when the migrants came in. I was warned 
t to let my students speak their native language and to reprimand students 
rho addressed me as "Mees Holley." I learned that those poor children in 
that school district were not high priority, as I compared the run down 
facility where I taught with the shiny new facility being built in ^ i 
neighborhood in which I lived. I also noted that the experienced 5th grade 
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teacher was "rewarded" with the higher group. 

When I entered the evaluation field in 1971 with my fresh degree 
in research via Title I program planning, I saw evaluation as an essential 
tool to remedy this lack of knowledge which seemed to be prevalent in our 
education of poor children. My tvjelve years in this role convinces me 
that this lack of knowledge is even more pervasive than I realized at that 
time. Intuitive decision-making is so pervasive in education that it 
borders on the unbelievable; with ramifications for the cost-effectiveness, 
efficiency, and quality of our educational system. Evaluation or data- 
based management in general are absolute necessities for school systems, 
whether we're talking about the education of poor children or running all 
the nation's schools. 

In Austin, we moved gradually during the Seventies from a "pro- 
gram evaluation,'' based on a hybrid discrepancy model, toward a "data-base," 
a generalized data-collection model from which systemwide studies, program 

evaluation, and Jin particular, planning information, can be drawn. We elim- 

i 

inated many program-specific data collection efforts in favor of selected 
targeted questions on systemwide surveys. Always, we have tried to move 
closer to the real, central concerns of the school district. 

I mention these things because I believe that in many school dis- 
tricts evaluation has met the test and local funding of evaluation has in- 
creased as federal funding has decreased; in others, however, the opposite 
is true. A crucial difference may be in how the evaluation unit serves 
the school district. 

In order to get some idea of how evaluation has fared in the Eight- 
ies, our office surveyed the 94 largest school districts (including Hawaii 
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and Puerto Rico) shown on the Directors of Research and Evaluation Netx^ork 
directory. The survey form is shown as Attachment 1. 

By our closing date we had received 60 responses, one of which 
could not be adequately interpreted. We received two after data analysis 
was complete for a total return of 66% and a useable return rate of 63% or 
59 responses used of a total of 94 possible. Mean lesponses are shown on 
the sample response form which is Attachment 1. 

It is of contextual interest to note that overall enrollments in 
the large cities are continuing to decrease. Only 12 of the 59 districts 
had enrollment increases and 7 of those increases occurred in districts 
with total enrollments below the median (1980-81) of 56,145. Only 4 dis- 
tricts with higher enrollments had. increases. 

As would be expected, the overall number of students eligible for 
Title I services also increased with the greatest number of increases 
coming in the districts above the median enrollment. Five of 29 high en- 
rollment districts had increases , but only three of 29 lower enrollment 
districts. Despite this overall increase in numbers eligible, the average 
number receiving services decreased although again eight districts did show 
increases. The increase in number served did not necessarily come in the 
cities with increases in number eligible. 

The average program budget decreased from 8.5 to 7.5 million dollars. 
The average Title I evaluation budget dropped sharply, from $122,233 to 
$100,622. Tne number of personnel also declined. The average number of 
clerical staff in evaluation went from 1.79 to 1.40; the average professional 
staff dropped from 1.75 to 1.50 and the average administrative staff from 
.45 to .37. 
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There was a smaller percentage decrease in overall federal funds 
for evaluation and it looks as though some of the slack for Title I may 
have been taken up by other sources of federal funds. The total estimated 
expenditure was $120,096 for 1982-83. 

The pleasant surprise, however, comes in the increase in local dol- 
lars for evaluation, up from $222,045 to $249,100. Evidently, local school 
systems do value the evaluation services they are receiving. Clerical and 
professional staff numbers are down somewhat, but locally supported admin- 
istrative staff is up. 

Responses to the question of whether there were changes in evalua- 
tion due to the reduction of federal funds were interesting, and responses 
have been appended ia the final pages of this paper. The responses do not 
indicate to me that most school systems are yet feeling drastic impact, but 
seve-ai indicate that they see the big cuts as due in the future. However, 
I believe we live in -^.n age where people demand information to such an ex- 
tent that we cannot go back. Evaluation is here to stay, and I believe we 
are learning to provide it more effectively and at lower cost. 



ATTACHMENT 1 

AUSTIN INDEPENDENT SCHOOL DISTRICT 
FEDERAL FUNDING SURVEY OFFICE OF RESEARCH AND EVALUATION 



PLEASE COMPLETE THE SURVEY BY GIVING THE FUND AMOUNTS AND NUMBERS OF STUDENTS 
FOR 1980-81 AND FOR 1982-83. NAME OF YOUR DISTRICT ir= 59 



iUiAL number or students In the district 


Means: 1981-82 


Means: 1982-83 


87,881 


85,453 


Numbejr of students ELIGIBLE, for TITLE l/CHAPTER 1 


23,890 


24,920 


Number of students SERVED by TITLE I/CHAPTER 1 


15,289 


14,086 


TOTAL TITLE l/CHAPTER 1 PROGRAM BUDGET (include 
evaluation funding) 


^ 8,572,240 


^ 7 ,595,024 


Evaluation funds from LOCAL/STATE . sources 


^ 222,045 


$ 249,100 


Number of LOCAL/STATE funded evaluation staff: 

CLERICAL 


3.20 


2.87 


PROFESSIONAL 


2.82 


2.75 


ADMmiSTRATIVE 


1.39 


1.43 


Evaluation funds from ALL FEDERAL sources 


$ 124,578 


$ 120,096 


Number of FEDERALLY funded evaluation staff : 

CLERICAL 


2.05 


1.60 


PROFESSIONAL 


2.61 


2.02 


ADMINISTRATIVE 


.46 


.42 


FvAluatlon funds frnm TTTT F T/rHAPTFR 1 


$ 122,233 


$ 100,622 


Number of TITLE I/CHAPTER 1 evaluation staff: 

CLERICAL 


1.79 


1.40 


PROFESSIONAL 


2.35 


1.90 


ADMINISTRATIVE 


.45 


.37 



HAVE THE REDUCTIONS IN FEDERAL FUNDS (SPECIFICALLY TITLE 1/ CHAPTER 1) MADE A 
DIFFERENCE IN EVALUATION ACTIVITIES, POLICIES, AND/OR PROCEDURES IN YOUR DIS- 
TRICT? 

SEE PAGES FOLLOWING. 



Please continue on the back. 



Freda Holley 6100 Guadalupe Box 79 

Austin Independent School District Austin, Texas 78752 



HAVE THE REDUCTIONS IN FEDERAL FUNDS (SPECIFICALLY TITLE l/CHAPTERl) MADE 
A DIFFERENCE IN EVALUATION ACTIVITIES, POLICIES, AND/OR PROCEDURES IN YOUR 
DISTRICT? 

Yes. We have to rely more on the personnel out at the school site to do 
the actual evaluation. 

Yes, two positions were already out from Ch. 1 evaluation and one more 
person will be cut this year. Process evaluation will suffer as a result, 
and the quantity of help we can provide instructional staff will, too. 

The loss of 1.5 Chapter 1 professionals has affected the number of Chap. 1 
site visitations and nas eliminated Chapter 1 process evaluation. 

No 5 it has not. 

All evaluation funds for personnel come from local sources. It is impos- 
sible to complete this form. The Director of Research writes up the 
portion of the Chapter 1 evaluation dealing with achievement. $6,000 was 
budgeted to cover test scoring for Chapter 1 students only. This has not 
changed in three years. Previously, a psychologist was hired by Chapter 1 
to supervise the entire process. Three years ago that position was elim- 
inated. 

Yes, from the inception of the Title I program in our district, the De- 
partment of Research and Evaluation was closely involved with the imple- 
mentation as well as the evaluation and monitoring of the program. This 
has changed, partly due to changes of organization in the district as well 
as funding changes. With the cutbacks anticipated in federal funds, the 
Research and Evaluation position of Title I Specialist (Ass't to Project 
Director) was changed to Title I Monitor (no longer an administrative func- 
tion) . Our Research and Evaluation Department is still responsible for 
monitoring and evaluating the Title l/Chapter 1 program, but is no longer 
as closely tied to planning and implementation. This is unfortunate as 
these functions are so closely intertwined. 

No. 

No. Although the total amount of overall support has dwindled, the com- 
mitment and support at the local level for good evaluation has remained 
constant . 

No. Reassignment of responsibilities has enables us to maintain the same 
level of evaluation activities. 

No. 

Reductions in Title I/Chapter 1 have had relatively little impact on the 
1982-83 evaluation activities. The major impact has come from local ad- 
ministrative decisions in two areas. The first decision area entails a 
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"freeze'' on positions — including externally-funded positions, (Our 
system was reorganized in August 1982, and our department x\ras "locked in" 
to a maximum number of positions •) As a consequence, we have approved 
funding for evaluation activities that cannot be fully implemented — i.e., 
the current number of assigned staff cannot absorb all of the funded eval- 
uation assignments. The second decision area, related to the first, centers 
on the nature of the proposed Chapter II Block grant activities and associ- 
ated evaluation activities. The Block Grant monies (approximately $1,000,000) 
are being used primarily for activities that will be documented (and reported) 
reducing the evaluator role (if any) to that of a consultant and reducing 
related evaluation monies to $8,973, a very small fraction of what would 
normally be assigned to a $1,800,000 funding. 

The evaluation process itself remains intact, however, the position Coor- 
dinator of Chapter I Evaluation has been reduced to a half-time position. 
The State Department of Education has reduced some data requirements for 
the evaluation, but that has not yet impacted the district's evaluation 
plan. 

The reduction in federal funds li lits the acquisition of materials and the 
hiring of necessary personnel. 

To some extent, however, the most serious losses of funds will be felt 
later in this state. 

No: except for 80-81,. an unusual year, all Title I/Chapter 1 evaluation 
has been very limited (Model A analysis of NRT data) . 

Reduction in federal funds to date have had only a minimal effect upon 
evaluation activities, policies, and procedures. However, since our bud- 
get for evaluation is a "bare-bones budg,.. t , " any further reduction in 
funds will have a significant effect upon all facets of evaluation. 

Testing and research service has been significantly reduced due to overall 
changes (reduction) in funding. 

There has not been a net reduction in federal funds in this district. Re- 
ductions in individual programs have not made a difference in evaluation 
activities, policies, and/or procedures. We are fortunate that both the 
Superintendent and the Board recognize the abilities and usefulness of the 

Research, Planning and Evaluation Division. i 

I 

No. 

Very little; slightly less on-site observations. 

The reductions in federal funds hc-^ve not affected evaluation activities 
in this district. 



No. Other than supplies and materials, evaluation services have been sup- 
plied through in-kind support by the local research office. 



Yes. Budget reductions have resulted in the elimination of the full-time 
evaluator. The position responsibilities are being shared by two members 
of the Chapter 1 administrative staff • 

It has af «cted the quantity of evaluation that can be provided both in 
the number of projects evaluated and the number of activities within each 
project • Greatest reduction has been in process evaluation which makes 
us wonder if we are evaluating non-events; 

No, 

No. This district has been weaning itself from dependency on federal funds 
for research and evaluation — most especially for staffing. We have prob- 
ably gotten more efficient* 

No, There have been many changes in evaluation activities, but these may 
be attributed to the administrative goal of site based management /evalua- 
tion. 

No, . 

Yes, elimination of summer school and reduction in staff and materials. 
Evaluation activities remained the same due to the Chapter 1 requirements. 

Not yet I We expect that 1984-85 will be the year of the big cut and re- 
sultant changes. 

It has caused us to eliminate selected reports and ,to reduce services to 
schools. 

No, the State Department of Education has requested that the local district 
continue using the national evaluation models and report as in previous 
years. 

No. Parent involvement and mature staff carry on an excellent program. 
No, 

The following differences from 1980-81 are noteable with respect to Chap- 
ter 1 evaluation: a) reduction of 0,5 (FTE) Chapter 1 evaluation staff; 
b) Elimination of a Chapter 1 early childhood program in 1982-83 that had 
served 3 and 4-year-olds since 1970, During the final year of this pro- 
gram, approximately $2200, in evaluation funds were budgeted; c) Use of an 
instrument for pre-post Chapter 1 evaluation that is administered to all 
students in the district (grade 4 and above) during the fall. This change, 
initiated in 1981-82, has resulted in a savings for cost of testing mater- 
ials etc, paid for by Chapter 1 funds. 

We have reduced the size, scope and quantity of reports that were formerly 
completed. We are making every attempt to put as much of our work on the 
computer as possible. 
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Too early to say decisively. 



No. Our allocation in 1980-81 was based on 1970 census figures whereas 
this year's allocation was based on the 1980 census figures. (This Dis- 
trict is in an area of rapid population growth.) 

Evaluation activities are relatively the same as far a's basic evaluation 
procedures. However, "extras" have been eliminated. 'For example: Test- 
ing has been streamlined so that school system testing data can be used 
where possible, rather than Chapter 1 testing children district-wide. 

Able to provide services to fewer children. 

Yes. Reduction in funds caused reduction of full time evaluation posi- 
tions. Now must go to the use of third party evaluators. Lose daily con-' 
tact of evaluator. Fund reductions have caused us to look for more eco- 
nomical V7ays to perform required evaluations, i.e. "piggy back" on to the 
district and/or state testing programs. 

Not to any great extent yet. 



Evaluation activities have been reduced to include mainly minimal require- 
ments. There no longer is the staff to explore evaluation questions which 
might be more relevant to education than those dealing with these require- 
ments . 

We spend less time engaging in "process" evaluation activities and miscel- 
laneous studies. Most of the time we worl^ on needs assessment, eligibility 
evaluation. \ 

The reduction in federal funds have affecoed: - the number of schools 
served; - the number of grades served; - me pupil/ teacher assistant ratio; 
- the professional Chapter I staff (reduced). 

Yes. However, the reductions have had ayminimal effect. By virtue of 
clerical reductions we have lost some fL^ibility for data reduction and 
can no longer accept short turnaround/requests for discretionary studies. 

The reductions in federal funds have resulted in a reduction in staff which 
has limited our ability to do timely process evaluation. The staff reduc- 
tions have not been proportional, to che school reductions; the frequency 
and duration of school visitations have consequently been reduced. 



With a 50 percent reduction in staff, the scope of the 1982-83 Chapter 1 
evaluation will be reduced and assistance in completing 1981-82 evaluation 
tasks is being provided by staff who are locally funded. Dependence on the 
systems 's computer services has been particularly problematic. 



No. 



Yes. Evaluation activities have been reduced 
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No reductions took place due to loss of Federal funds. The reduction of 
1 professional staff member was due to attrition and the position \-ms not 
refilled due to anticipated cut backs associated with ECIA legislation. 
When the district's funds were not cut sever ly, the state and local n;one- 
tary reduction prohibited the refilling of this position. 

No. 
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